This invention pertains to the input interface to a virtual environment (VE), which allows the user to manipulate objects in the environment, and the output interface from the virtual environment, which allows the user to view objects in the environment.
Natural interfaces such as speech and gesture promise to be the best input interfaces for use in such virtual environments, replacing current desktop-oriented devices like the keyboard and mouse which are not as appropriate for virtual environment systems. However, little is known about how to integrate such advanced interfaces into virtual environment systems. Commercial tools exist to construct 3-D "worlds" using objects described with the Virtual Reality Modeling Language (VRML) standard. This constitutes the visual interface to the virtual environment. However, no such tools exist to augment these objects with natural input interfaces most appropriate to immersive visualization, such as speech.
Research prototypes have demonstrated the power of natural interfaces in virtual environments, but this work has not generated methods for constructing natural interfaces to virtual environments that are comparable in power to common methods for building visual interfaces. For example, previous speech interfaces to virtual environments consist of specialized "speech aware" tools within the virtual environment (see for example Jason Leigh et al. article entitled "Multi-Perspective Collaborative Design in Persistent Networked Virtual Environments," in Proceedings of IEEE Virtual Reality Annual Symposium, VRAIS'96, pages 253-260, Santa Clara, Calif., March 1996), and expert systems for mapping voice commands to actions within a specific domain (see for example Mark Billinghurst et al., "Adding Intelligence to the Interface," Proceedings of IEEE Virtual Reality Annual International Symposium, pages 168-175, Santa Clara. Calif.). In both cases, the interface is separate from the objects it manipulates; if the configuration of the objects in the virtual environment changes, the interface must be reconfigured or re-trained. Thus, neither is applicable to rapid construction of flexible virtual environments for general use.
The purpose of this invention is to enable virtual environment creators to embed natural interfaces directly into the objects of the virtual environment. This permits rapid creation of virtual environments, intuitive interaction with virtual objects. and straightforward interface reconfiguration.